Transfer Meta-Learning: Information- Theoretic Bounds and Information Meta-Risk Minimization

Meta-learning automatically infers an inductive bias by observing data from a number of related tasks. The is encoded hyperparameters that determine aspects the model class or training algorithm, such as initialization learning rate. assumes tasks belong to task environment, and are drawn same environment both during meta-training meta-testing. This, however, may not hold true in practice. In this paper, we introduce problem transfer meta-learning, which target meta-testing differ source observed meta-training. Novel information-theoretic upper bounds obtained on meta-generalization gap, measures difference between loss, available at meta-learner, average loss meta-test new, randomly selected, environment. first bound, captures meta-environment shift environments via KL divergence distributions. second, PAC-Bayesian third, single-draw account for log-likelihood ratio Furthermore, two meta-learning solutions introduced. For first, termed Empirical Meta-Risk Minimization (EMRM), derive optimality gap. referred Information (IMRM), minimizing bound. IMRM shown experiments potentially outperform EMRM.